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FOREWORD 


During  1963  -  1964  period,  the  ESD  Reliability/Maintain¬ 
ability  staff  conducted  a  series  of  lectures  and  briefings  on 
R/M.  Several  lectures  dealt  with  equipment  MTBF  demonstration 
problems . 

This  TDR  summarizes  a  discussion  conducted  on  the  applications 
of  the  Cummulative  Poisson  Function  to  MTBF  demonstration.  The 
ideas  produced  during  this  and  subsequent  discussions  are  re¬ 
flected  in  ESD-TDR-64-616. 
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SECTION  I 


INTRODUCTION 


The  specification  of  a  minimum  MTBF  for  an  equipment  introduces 
the  problem  of  demonstration.  At  ESD,  we  are  usually  operating  with 
at  least  four  constraints: 

1.  Equipment  MTBFs  are  in  the  order  of  500  hours  or  more. 

2.  Limited  numbers  (usually  one  or  two)  of  articles  are 
purchased. 

3-  Delivery  schedules  are  compressed. 

4.  A  finite  number  of  dollars  are  available. 

With  such  constraints,  the  present  reliability  demonstration  model  de¬ 
fined  in  MIL-R-26474  does  not  offer  a  satisfactory  solution  to  the 
demonstration  problem.  This  model  could  suspend  decisions  on  compliance 
to  contractual  MTBFs  for  as  much  as  a  year  (some  of  our  programs  require 
delivery  within  8  to  10  months  from  contract  award) .  The  result,  when 
we  attempt  to  apply  and  enforce  MIL-R-26474  demonstration  criteria, 
is  usually  a  request  for  waiver  on  the  part  of  a  contractor  and  the  de¬ 
velopment  of  "home-made"  decision  criteria  as  to  the  acceptability  of 
equipment.  These  criteria  are  usually  non-quant itatlve  and  involve 
personal  expressions;  such  as,  "I  think  we  should  accept  (or  reject)". 
What  is  needed,  of  course,  is  a  quantitative  decision  rule  or  model 
which  has  at  least  the  following  essential  properties: 

1.  It  is  understood  by  both  the  customer  (ESD)  and  contractor. 

2.  Quantifies  the  risks  involved  in  decisions;  i.e.,  the  prob¬ 
ability  of  rejecting  equipments  which  have  achieved  the 
contractual  MTBF  and  the  probability  of  accepting  equipments 
which  have  not  achieved  the  contractual  MTBF. 

3-  Allows  scheduling  and  cost  analysis  within  the  overall  con¬ 
straints  of  the  program. 

The  ESD  is  beginning  to  apply  a  simple  decision  rule  which  we  feel 
satisfies  the  above  essential  properties.  This  rule  states  that  equip¬ 
ment  will  be  planned  to  be  exercised  for  a  specific  amount  of  time  (such 
as  one  multiple  of  contractual  MTBF)  and  during  this  time  a  specific 
number  of  failures  will  be  allowed.  If  this  number  is  exceeded,  the 
equipment  will  be  declared  unacceptable. 
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For  example,  take  the  case  where  test  time  is  set  equal  to  0  ,  the 
contractual  MTBF.  If  c  is  set  equal  to  zero,  a  contractor  would  have 
a  39 $  chance  that,  even  if  his  equipment  had  an  MTBF  equal  to  20  ,  the 
equipment  would  be  rejected.  On  the  other  hand,  the  customer  has  a  l4$ 
chance  of  accepting  an  equipment  which  has  an  MTBF  equal  to  l/2  0^.  Such 
a  rule,  while  protecting  the  customer,  would  probably  be  unacceptable  to 
a  contractor.  To  minimize  his  risk,  c  could  be  set  equal  to  one.  If 
his  equipment  had  an  MTBF  equal  to  2  0  ,  his  risk  (probability  of  rejec¬ 
tion)  would  now  be  8$,  but  the  customer  would  have  a  42$  chance  of  accept¬ 
ing  an  equipment  which  has  an  MTBF  equal  to  l/2  9^ 

l 

This  simple  arithmetic  illustrates  the  need  for  clearly  setting  forth 
and  obtaining  agreement  on  a  value  of  c.  We  expect  that  some  discussion 
on  this  matter  will  take  place  in  technical  proposals  and  in  contract 
negotiations.  The  point  of  consideration  in  these  discussions  is  the 
fact  that  both  parties  are  aware  of  each  others'  risks. 

If  by  accident,  ESD  cites  MIL-R-26474  techniques  in  an  RFP,  we  expect 
bidders  to  take  us  to  task  when  they  discover  that  such  techniques  are 
unrealistic  for  the  program  under  consideration.  We  also  hope  that  they 
will  consider  and  offer  suggestions  for  a  fixed  test  time,  allowable  num¬ 
ber  of  failures,  type  of  demonstration  (or  another  suitable  technique). 
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SECTION  II 


THE  CUMMULATIVE  POISSON  MODEL 


The  selection  of  an  allowable  number  of  failures  (c)  is  a  matter  of 
concern  not  only  for  ESD  but  also  a  contractor.  In  the  first  place, 
as  indicated  in  Section  I,  c  is  dependent  on  the  number  of  programmed 
test  hours.  Usually,  the  greater  the  test  time,  the  greater  the  value 
taken  for  c.  This  results  from  the  consideration  that  as  c  increases, 
the  probability  of  acceptance  increases  and,  therefore,  the  contractor's 
or  producer's  statistical  risk  decreases. 

However,  for  a  given  c  value  and  test  time,  a  contractor  can  reduce 
his  statistical  risk  by  increasing  the  value  of  the  ratio  of  "true"  MTBF 
capability  to  contractual  MTBF,  (0^/0  )•  Estimates  of  0^  are  provided 
during  the  design  of  an  equipment  in  ierms  of  MTBF  predictions. 

These  few  statistical  concepts  can  be  illustrated  with  a  family  of 
Poisson  operating  characteristic  functions.  A  purpose  of  this  discussion 
is  to  indicate  a  method  by  which  such  functions  can  be  constructed. 

The  Cummulative  Poisson  expression  for  c  or  less  failures  in  t  hours 
of  operation,  given  an  MTBF  equal  to  0^.,  is 


EQ.l  P(x  S  c)  »  e_t/0t  (t/0t)X/x: 

x  =  0,  1,  2,  ... 

To  illustrate  the  application  of  the  above  equation,  assume  0^.  =  1000 
hoars  and  that  it  has  been  decided  to  test  for  one  multiple  of  0^.  The 
probability  of  exactly  zero  failures  during  the  test  is, 


EQ.2  P(x  =  0)  =  e-1000/l000 

=  0.37 


(iooo/iooo)°/o: 


The  probability  of  one  or  less  failures  during  the  test  is, 

EQ. 3  P(x  S  1)  =  XiU  e“1000/i000  ( 1000 / 1000) X/ x i 

x~u 

=  0.74 


If  the  MTBF  is  2000  hours,  then,  during  a  1000  hour  test,  the  prob¬ 
ability  of  one  or  less  failures  is  0.$2.  If,  on  the  other  hand,  the  MTBF 
is  500  hours,  the  probability  of  one  or  less  failures  is  0.42. 


3 


The  practical  significance  of  the  above  numbers  is  interesting. 

Assume  that  the  demonstration  decision- rule  for  a  1000  hour  MTBF  stated: 
"test  for  1000  hours  of  satisfactory  operation  with  no  more  than  one 
failure".  A  contractor  would  have  a  0.08  statistical  chance  that  even 
if  his  equipment  had  a  2000  hour  MTBF  capability  it  would  be  rejected 
(experience  two  or  more  failures)  by  the  decision-rule.  On  the  other 
hand,  with  an  MTBF  capability  of  500  hours  (l/2  the  MTBF  requirement)  a 
contractor  has  a  0.42  statistical  probability  that  his  equipment  still 
will  be  accepted  by  the  decision-rule. 

If  the  latter  statistical  probability  is  disturbing  to  ESD,  a  re¬ 
duction  in  the  permissable  number  of  failures  from  c  =  1  to  c  =  0  would 
allow  only  a  0.l4  probability  that  equipment  with  l/2  the  MTBF  require¬ 
ment  would  be  accepted.  On  the  other  hand,  to  have  only  a  0.10  probability 
of  rejection,  a  contractor's  equipment  would  have  to  have  a  10,000  hour 
MTBF  capability.  To  design  this  capability  may  be  either  technically 
impossible  or  economically  unattractive  to  both  ESD  and  a  contractor  and 
a  further  analysis  of  the  demonstration  decision-rule  would  be  required. 

This  arithmetic  illustrates  an  approach  to  the  development  of  Poisson 
operating  characteristic  functions.  Such  functions  relate  the  probability 
of  acceptance  to  the  ratio  ( Q./Q  )  for  values  of  c. 
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SECTION  III 


SOME  SAMPLE  POISSON  OPERATING 
CHARACTERISTIC  FUNCTIONS 


Table  1  presents  values  of  three  Poisson  operating  characteristic 
functions  when  test  time  is  set  equal  to  one  multiple  of  contractual 
MTBF.  Figure  1  is  a  graph  of  these  three  functions.  As  the  functions 
indicate,  the  most  favorable  decision- rule,  of  the  three  presented  for 
ESD,  occurs  when  c  is  set  equal  to  zero.  On  the  other  hand,  as  expected, 
the  most  favorable  decision-rule  to  a  contractor  occurs  when  c  is  set 
equal  to  two. 

The  reasoning  behind  the  above  observations  is  apparent  from  the 
probability  of  acceptance  values  in  Table  1.  ESD  is  concerned  with  the 
probability  of  accepting  equipment  which  is  "unsatisfactory",  i.e., 
probability  of  accepting  when  -  1*  These  probabilities  decrease 

either  as  the  ratio  decreases  and/or  the  value  of  c  decreases. 

For  example:  if  one  failure  is  permitted  in  a  test  of  one  multiple  of 
MTBF  duration,  the  probability  of  accepting  equipment  which  has  l/2  the 
MTBF  requirement  is  0.42;  when  c  is  set  equal  to  zero,  this  probability 
decreases  to  0.l4.  The  probabilities  of  accepting  equipment  which  has 
l/4  the  MTBF  requirement  when  c  =  1  and  c  =  0  are  0.10  and  0.02,  re¬ 
spectively. 

A  compromise  decision-rule,  as  indicated  in  Figure  1,  would  involve 
setting  c  =  1.  This  means  that  the  probability  of  rejection  for  all 
values  of  -  1  never  exceeds  0.26.  When  the  ratio  =  2,  there 

is  only  a  0.08  chance  that  two  or  more  failures  will  occur.  On  the  other 
hand,  there  is  still  a  0.10  statistical  probability  that,  when  0^/0^  -  l/4, 
one  or  less  failures  will  oc.cur. 

If  it  were  possible  to  extend  test  duration  to,  say,  two  multiples  of 
contractual  MTBF,  a  different  family  of  Poisson  operating  characteristic 
functions  could  be  generated.  Some  values  of  these  functions  are  given 
in  Table  2.  A  graphical  presentation  is  given  in  Figure  2.  Table  1  and  2 
values  indicate  the  effect  on  statistical  probability  of  acceptance  when 
test  duration  is  increased  and  c  is  held  constant.  For  example,  if  c  =  1, 
the  statistical  probability  of  accepting  equipment  which  has  an  MTBF  equal 
to  l/2  the  contractual  value  is  0.42,  for  the  case  when  test  duration  is 
equal  to  one  multiple  of  required  MTBF,  but  is  only  0.10,  when  test  dur¬ 
ation  is  two  times  the  contractual  value. 

Additional  computations  can  be  performed  for  different  values  of  test 

duration.  To  illustrate  the  ease  by  which  this  may  be  accomplished,  a 

detailed  example  is  presented  in  Section  V  for  the  case  when  test  duration 

is  equal  to  three  multiples  of  contractual  MTBF  and  0,/0  =  2. 

z  c 
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Table  1 


Some  Values 

of  Poisson  Operating  Characteristic  Functions 

When  Test 

Duration  Equals  The 

Contractual  MTBF  Value 

’t/^c 

c  =.  0 

c  =  1 

c  =  2 

iA 

0.02 

0.10 

0.26 

1/2 

0.l4 

0.42 

0.70 

1 

0.37 

0.74 

0.87 

3/2 

0.51 

O.85 

0.97 

2 

0.6l 

0.92 

1 

Figure  1 


Some  Poisson  Operating  Characteristic  Functions 
When  Test  Duration  Equals  One  Multiple  of  MTBF  (@c) 


"True"  MTBF  /  Contractual.  MTBF 


Table  2 


Some  Values  of  Poisson  Operating  Characteristic  Functions 

When  Test  Duration  Equals  Twice  The  Contractual  MTBF  Value 


t/ec 

c  =  0 

c  =  1 

c  =  2 

c  =  ; 

1/4 

0.0003 

0.0024 

0.01 

0.04 

1/2 

0.02 

0.10 

0.26 

0.47 

1 

0.l4 

0.42 

0.70 

0.89 

3/2 

0.27 

0.62 

O.85 

0.95 

2 

0.37 

0.74 

0.92 

O.98 

J 
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SECTION  IV 


SUMMARY 


To  assist  in  the  selection  of  a  c  value  for  a  demonstration  problem 
which  satisfies  the  constraints  discussed  in  Section  I,  it  i,s  suggested 
that  Poisson  operating  characteristic  functions  be  developed.  Such 
functions  present,  for  selected  c  and  test  duration  values,  the  proba¬ 
bilities  of  accepting  equipment  which  has  either  met,  failed  to  meet, 
or  exceeded  an  MTBF  requirement. 

For  ease  of  computational  presentation,  only  integral  multiples  of 
contractual  MTBF  have  been  used  in  this  discussion  in  assigning  a  test 
duration  value.  Obviously,  in  a  given  demonstration  problem,  other 
numbers  can  be  used.  A  selection  of  a  test  duration  value  depends  on 
such  considerations  as  program  schedules,  quantities  of  equipment  avail¬ 
able  for  test,  availability  of  test  dollars,  etc. 

It  is  recognized  that  the  Cummulative  Poisson  Function  is  only  one 
statistical  model  presently  available  for  application  on  equipment  MTBF 
demonstration  problems  and  that  the  Poisson  approach,  with  test  duration 
held  to  a  minimum,  does  not  give  a  high  statistical  confidence  that  the 
contractual.  MTBF  has  been  achieved  or  exceeded.  But,  with  the  constraints 
of  Section  I,  it  at  least  satisfies  the  need  for  a  quantitative  decision- 
rule.  Furthermore,  from  the  reliability  engineering  management  viewpoint, 
a  one  or  two  multiple  of  MTBF  test  with,  say,  no  more  than  one  failure  is 
not  considered  to  be  a  unsatisfactory  procedure.  Rather,  it  could  cause 
contractor  management  to  note  the  need  for  delivering  equipment  for  dem¬ 
onstration  which  has  been  designed  and  manufactured  for  reliable  operation, 
especially,  when  supplemented  by  a  retest  penalty  clause  which  places  the 
complete  cost  burden  of  failing  the  original  demonstration  on  a  contractor. 

Furthermore,  the  budgeting  problem  for  a  fixed  test  time  reliability 
demonstration  is  easier  to  solve  than  one  influenced  by  a  variable  test 
time. 

The  statistical  ideas  in  this  discussion  can  be  extended  to  establish 
a  more  comprehensive  view  of  the  reliability  demonstration  problem.  A 
task  to  accomplish  this  objective  has  been  undertaken  by  the  Technical  Re¬ 
quirements  and  Standards  Office. 
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SECTION  V 


SAMPLE  COMPUTATION 


P  (c) 


e't/8t  (t/et)c 

c.f 


Assume  the  test  duration  is  to  be  fixed  at  three  times  the  contractual 

MTBF;  i.e.,  30  .  If  0.  is  two  times  the  contractual  MTBF, 
c  u 


P  (c) 


(3ec/29c)C 


-1-5 

P  (0)  =  e  =  0.22 

-1-5  x 

p  (1)  =  - - 1,  =  0.33 

-1.5  2 

P  (2)  =  - 27 - '  =  0.25 

p  (3)  =  1 - 3v  ■  =0.12 

Therefore, 

P  (  1  or  less  )  =  0.22  +  0-33  =  O.55 

P  (  2  or  less  )  =  0-55  +  0.25  =  0.80 

P  (  3  or  less  )  =  0.80  +  0.12  '=  0.92 
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